2 Administering Oracle Big Data Appliance

2.1.1 Using the Enterprise Manager Web Interface

After opening Oracle Enterprise Manager web interface, logging in, and selecting a target cluster, you can drill down into these primary areas:

• InfiniBand network: Network topology and status for InfiniBand switches and ports. See Figure 2-1.

• Hadoop cluster: Software services for HDFS, MapReduce, and ZooKeeper.

• Oracle Big Data Appliance rack: Hardware status including server hosts, Oracle Integrated Lights Out Manager (Oracle ILOM) servers, power distribution units (PDUs), and the Ethernet switch.

Figure 2-1 shows a small section of the cluster home page.

Figure 2-1 YARN Page in Oracle Enterprise Manager

Description of "Figure 2-1 YARN Page in Oracle Enterprise Manager"

To monitor Oracle Big Data Appliance using Oracle Enterprise Manager: 

1. Download and install the plug-in. See Oracle Enterprise Manager System Monitoring Plug-in Installation Guide for Oracle Big Data Appliance.

2. Log in to Oracle Enterprise Manager as a privileged user.

3. From the Targets menu, choose Big Data Appliance to view the Big Data page. You can see the overall status of the targets already discovered by Oracle Enterprise Manager.

4. Select a target cluster to view its detail pages.

5. Expand the target navigation tree to display the components. Information is available at all levels.

6. Select a component in the tree to display its home page.

7. To change the display, choose an item from the drop-down menu at the top left of the main display area.

2.1.2 Using the Enterprise Manager Command-Line Interface

The Enterprise Manager command-line interface (emcli) is installed on Oracle Big Data Appliance along with all the other software. It provides the same functionality as the web interface. You must provide credentials to connect to Oracle Management Server.

To get help, enter emcli help.

2.2.1 Monitoring the Status of Oracle Big Data Appliance

In Cloudera Manager, you can choose any of the following pages from the menu bar across the top of the display:

• Home: Provides a graphic overview of activities and links to all services controlled by Cloudera Manager. See Figure 2-2.

• Clusters: Accesses the services on multiple clusters.

• Hosts: Monitors the health, disk usage, load, physical memory, swap space, and other statistics for all servers in the cluster.

• Diagnostics: Accesses events and logs. Cloudera Manager collects historical information about the systems and services. You can search for a particular phrase for a selected server, service, and time period. You can also select the minimum severity level of the logged messages included in the search: TRACE, DEBUG, INFO, WARN, ERROR, or FATAL.

• Audits: Displays the audit history log for a selected time range. You can filter the results by user name, service, or other criteria, and download the log as a CSV file.

• Charts: Enables you to view metrics from the Cloudera Manager time-series data store in a variety of chart types, such as line and bar.

• Backup: Accesses snapshot policies and scheduled replications.

• Administration: Provides a variety of administrative options, including Settings, Alerts, Users, and Kerberos.

Figure 2-2 shows the Cloudera Manager home page.

Figure 2-2 Cloudera Manager Home Page

Description of "Figure 2-2 Cloudera Manager Home Page"

2.2.2 Performing Administrative Tasks

As a Cloudera Manager administrator, you can change various properties for monitoring the health and use of Oracle Big Data Appliance, add users, and set up Kerberos security.

To access Cloudera Manager Administration: 

1. Log in to Cloudera Manager with administrative privileges.

2. Select the Administration tab.

2.2.3 Managing Services With Cloudera Manager

Cloudera Manager provides the interface for managing these services:

• HDFS

• Hive

• Hue

• Oozie

• YARN

• ZooKeeper

You can use Cloudera Manager to change the configuration of these services, stop, and restart them. Additional services are also available, which require configuration before you can use them. See "Unconfigured Software."

2.3.1 Monitoring MapReduce Jobs

You can monitor MapReduce jobs using the resource manager interface.

To monitor MapReduce jobs: 

• Open a browser and enter a URL like the following:

  http://bda1node03.example.com:8088
  

  In this example, bda1 is the name of the rack, node03 is the name of the server where the YARN resource manager runs, and 8088 is the default port number for the user interface.

Figure 2-3 shows the resource manager interface.

Figure 2-3 YARN Resource Manager Interface

Description of "Figure 2-3 YARN Resource Manager Interface"

2.3.2 Monitoring the Health of HDFS

You can monitor the health of the Hadoop file system by using the DFS health utility on the first two nodes of a cluster.

To monitor HDFS: 

• Open a browser and enter a URL like the following:

  http://bda1node01.example.com:50070
  

  In this example, bda1 is the name of the rack, node01 is the name of the server where the dfshealth utility runs, and 50070 is the default port number for the user interface.

Figure 2-3 shows the DFS health utility interface.

Figure 2-4 DFS Health Utility

Description of "Figure 2-4 DFS Health Utility"

2.5.1 Software Components

These software components are installed on all servers in the cluster. Oracle Linux, required drivers, firmware, and hardware verification utilities are factory installed. All other software is installed on site. The optional software components may not be configured in your installation.

Base image software: 

• Oracle Linux 6.4 (upgrades stay at 5.8) with Oracle Unbreakable Enterprise Kernel version 2 (UEK2)

• Java HotSpot Virtual Machine 7 version 25 (JDK 7u25)

• Oracle R Distribution 3.0.1-2

• MySQL Database 5.5.35 Advanced Edition

• Puppet, firmware, Oracle Big Data Appliance utilities

• Oracle InfiniBand software

Mammoth installation: 

• Cloudera's Distribution including Apache Hadoop Release 5 (5.0) including:

  • Apache Hive 0.12

  • Apache HBase

  • Apache Sentry

  • Apache Spark

  • Cloudera Impala

  • Cloudera Search 1.2.0

• Cloudera Manager Release 5 (5.0) including:

  • Cloudera Navigator

• Oracle Database Instant Client 12.1

• Oracle NoSQL Database Community Edition or Enterprise Edition 12c Release 1 Version 3.0.5 (optional)

• Oracle Big Data Connectors 3.0 (optional):

  • Oracle SQL Connector for Hadoop Distributed File System (HDFS)

  • Oracle Loader for Hadoop

  • Oracle Data Integrator Agent 11.1.1.7.0

  • Oracle XQuery for Hadoop

  • Oracle R Advanced Analytics for Hadoop

Figure 2-6 shows the relationships among the major components.

Figure 2-6 Major Software Components of Oracle Big Data Appliance

Description of "Figure 2-6 Major Software Components of Oracle Big Data Appliance"

2.5.2 Unconfigured Software

Your Oracle Big Data Appliance license includes all components in Cloudera Enterprise Data Hub Edition. All CDH components are installed automatically by the Mammoth utility. Do not download them from the Cloudera website.

However, you must use Cloudera Manager to add the following services before you can use them:

• Apache Flume

• Apache HBase

• Apache Spark

• Apache Sqoop

• Cloudera Impala

• Cloudera Navigator

• Cloudera Search (Solr)

To add a service: 

1. Log in to Cloudera Manager as the admin user.

2. On the Home page, expand the cluster menu in the left panel and choose Add a Service to open the Add Service wizard. The first page lists the services you can add.

3. Follow the steps of the wizard.

You can find the RPM files on the first server of each cluster in /opt/oracle/BDAMammoth/bdarepo/RPMS/noarch.

2.6.1 Where Do the Services Run on a Single Rack?

Table 2-1 identifies the services in CDH clusters configured within a single rack, including starter racks and clusters with six nodes. Node01 is the first server in the cluster (server 1, 7, or 10), and nodenn is the last server in the cluster (server 6, 9, 12, or 18).

^Footnote 1 Starter racks and six-node clusters only, with reduced allocated resources

2.6.2 Where Do the Services Run on a Multirack Cluster?

When multiple racks are configured as a single CDH cluster, some critical services are installed on the second rack.

Table 2-2 identifies the location of services on the first rack of a multirack cluster.

^Footnote 1 nn includes the servers in additional racks.

Table 2-3 shows the service locations in the second rack of a multirack cluster.

^Footnote 1 Starter racks and six-node clusters only, with reduced allocated resources

2.6.3 About MapReduce

Yet Another Resource Negotiator (YARN) is the default version of MapReduce, starting with Oracle Big Data Appliance 3.0. Oracle recommends YARN for production systems. MapReduce 1 (MRv1) is supported for backward compatibility, but you must install it manually. Do not run both versions of MapReduce (YARN and MRv1) on the same cluster.

MapReduce applications developed using MapReduce 1 may require recompilation to run under YARN.

The ResourceManager performs all resource management tasks. An MRAppMaster performs the job management tasks. Each job has its own MRAppMaster. The NodeManager has containers that can run a map task, a reduce task, or an MRAppMaster. The NodeManager can dynamically allocate containers using the available memory. This architecture results in improved scalability and better use of the cluster than MRv1.

YARN also manages resources for Spark and Impala.

MRv1 manages MapReduce jobs exclusively. The JobTracker combines the functions of the ResourceManager and MRAppMaster. TaskTrackers perform the function of the NodeManagers.

2.6.4 Automatic Failover of the NameNode

The NameNode is the most critical process because it keeps track of the location of all data. Without a healthy NameNode, the entire cluster fails. Apache Hadoop v0.20.2 and earlier are vulnerable to failure because they have a single name node.

Cloudera's Distribution including Apache Hadoop Version 4 (CDH5) reduces this vulnerability by maintaining redundant NameNodes. The data is replicated during normal operation as follows:

• CDH maintains redundant NameNodes on the first two nodes of a cluster. One of the NameNodes is in active mode, and the other NameNode is in hot standby mode. If the active NameNode fails, then the role of active NameNode automatically fails over to the standby NameNode.

• The NameNode data is written to a mirrored partition so that the loss of a single disk can be tolerated. This mirroring is done at the factory as part of the operating system installation.

• The active NameNode records all changes to the file system metadata in at least two JournalNode processes, which the standby NameNode reads. There are three JournalNodes, which run on the first three nodes of each cluster.

• The changes recorded in the journals are periodically consolidated into a single fsimage file in a process called checkpointing.

Figure 2-7 shows the relationships among the processes that support automatic failover of the NameNode.

Figure 2-7 Automatic Failover of the NameNode on Oracle Big Data Appliance

Description of "Figure 2-7 Automatic Failover of the NameNode on Oracle Big Data Appliance"

2.6.5 Automatic Failover of the ResourceManager

The ResourceManager allocates resources for application tasks and application masters across the cluster. Like the NameNode, the ResourceManager is a critical point of failure for the cluster. If all ResourceManagers fail, then all jobs stop running. Oracle Big Data Appliance 3.0 supports ResourceManager High Availability in Cloudera 5 to reduce this vulnerability.

CDH maintains redundant ResourceManager services on node03 and node04. One of the services is in active mode, and the other service is in hot standby mode. If the active service fails, then the role of active ResourceManager automatically fails over to the standby service. No failover controllers are required.

Figure 2-7 shows the relationships among the processes that support automatic failover of the ResourceManager.

Figure 2-8 Automatic Failover of the ResourceManager on Oracle Big Data Appliance

Description of "Figure 2-8 Automatic Failover of the ResourceManager on Oracle Big Data Appliance"

2.6.6 Map and Reduce Resource Configuration

The NodeManager services are allocated a fixed amount of memory and CPU core voltage (Vcore), as shown in Table 2-4.

Each node in a cluster has a maximum number of map and reduce tasks that are allowed to run simultaneously. Table 2-5 shows the default configuration of resources for the MapReduce service on Oracle Big Data Appliance X4-2 and X3-2.

^Footnote 1 9 or more nodes

Table 2-6 shows the default configuration of resources for the MapReduce service on Oracle Big Data Appliance Sun Fire X4270 M2-based racks.

^Footnote 1 9 or more nodes

2.7.1 Logical Disk Layout

Each server has 12 disks. The critical operating system is stored on disks 1 and 2.

Table 2-7 describes how the disks are partitioned.

2.7.2 Critical and Noncritical Nodes

Critical nodes are required for the cluster to operate normally and provide all services to users. In contrast, the cluster continues to operate with no loss of service when a noncritical node fails.

On single-rack clusters, the critical services are installed initially on the first four nodes of the cluster. The remaining nodes (node05 up to node18) only run noncritical services. If a hardware failure occurs on one of the critical nodes, then the services can be moved to another, noncritical server. For example, if node02 fails, then you might move its critical services node05. Table 2-1 identifies the initial location of services for clusters that are configured on a single rack.

In a multirack cluster, some critical services run on the first server of the second rack. See "Where Do the Services Run on a Single Rack?."

Moving a critical node requires that all clients be reconfigured with the address of the new node. The other alternative is to wait for the failed server to be repaired. You must weigh the loss of services against the inconvenience of reconfiguring the clients.

^Footnote 1 In multirack clusters, this service is initially installed on the first server of the second rack.

2.7.3 First Namenode

One instance of the NameNode initially runs on node01. If this node fails or goes offline (such as a restart), then the second NameNode (node02) automatically takes over to maintain the normal activities of the cluster.

Alternatively, if the second NameNode is already active, it continues without a backup. With only one NameNode, the cluster is vulnerable to failure. The cluster has lost the redundancy needed for automatic failover.

The puppet master also runs on this node. The Mammoth utility uses Puppet, and so you cannot install or reinstall the software if, for example, you must replace a disk drive elsewhere in the rack.

2.7.4 Second NameNode

One instance of the NameNode initially runs on node02. If this node fails, then the function of the NameNode either fails over to the first NameNode (node01) or continues there without a backup. However, the cluster has lost the redundancy needed for automatic failover if the first NameNode also fails.

The MySQL backup database also runs on this node. MySQL Database continues to run, although there is no backup of the master database.

2.7.5 First ResourceManager

One instance of the ResourceManager initially runs on node03. If this node fails or goes offline (such as a restart, then the second ResourceManager (node04) automatically takes over to distribute MapReduce tasks to specific nodes across the cluster.

Alternatively, if the second ResourceManager is already active, it continues without a backup. With only one ResourceManager, the cluster is vulnerable to failure. The cluster has lost the redundancy needed for automatic failover.

These services are also disrupted:

• Cloudera Manager: This tool provides central management for the entire CDH cluster. Without this tool, you can still monitor activities using the utilities described in "Using Hadoop Monitoring Utilities".

• MySQL Master Database: Cloudera Manager, Oracle Data Integrator, Hive, and Oozie use MySQL Database. The data is replicated automatically, but you cannot access it when the master database server is down.

2.7.6 Second ResourceManager

One instance of the ResourceManager initially runs on node04. If this node fails, then the function of the ResourceManager either fails over to the first ResourceManager (node03) or continues there without a backup. However, the cluster has lost the redundancy needed for automatic failover if the first ResourceManager also fails.

These services are also disrupted:

• Oracle Data Integrator: This service supports Oracle Data Integrator Application Adapter for Hadoop. You cannot use this connector when the ResourceManager node is down.

• Hive: Hive provides a SQL-like interface to data that is stored in HDFS. Most of the Oracle Big Data Connectors can access Hive tables, which are not available if this node fails.

• Hue: This administrative tool is not available when the ResourceManager node is down.

• Oozie: This workflow and coordination service runs on the ResourceManager node, and is unavailable when the node is down.

2.7.7 Noncritical Nodes

The noncritical nodes (node05 to node18) are optional in that Oracle Big Data Appliance continues to operate with no loss of service if a failure occurs. The NameNode automatically replicates the lost data to always maintain three copies. MapReduce jobs execute on copies of the data stored elsewhere in the cluster. The only loss is in computational power, because there are fewer servers on which to distribute the work.

2.8.1 Prerequisites

You must have root access. Passwordless SSH must be set up on the cluster, so that you can use the dcli utility.

To ensure that passwordless-ssh is set up: 

1. Log in to the first node of the cluster as root.

2. Use a dcli command to verify it is working. This command should return the IP address and host name of every node in the cluster:

   # dcli -C hostname
   192.0.2.1: bda1node01.example.com
   192.0.2.2: bda1node02.example.com
        .
        .
        .
   

3. If you do not get these results, then set up dcli on the cluster:

   # setup-root-ssh -C
   

2.8.2 Stopping Oracle Big Data Appliance

Follow these procedures to shut down all Oracle Big Data Appliance software and hardware components.

Task 1   Stopping All Managed Services

Use Cloudera Manager to stop the services it manages, including flume, hbase, hdfs, hive, hue, mapreduce, oozie, and zookeeper.

1. Log in to Cloudera Manager as the admin user.

   See "Managing CDH Operations Using Cloudera Manager".

2. In the Status pane of the opening page, expand the menu for the cluster and click Stop, and then click Stop again when prompted to confirm. See Figure 2-9.

   To navigate to this page, click the Home tab, and then the Status subtab.

3. On the Command Details page, click Close when all processes are stopped.

4. In the same pane under Cloudera Management Services, expand the menu for the mgmt service and click Stop.

5. Log out of Cloudera Manager.

Figure 2-9 Stopping HDFS Services

Description of "Figure 2-9 Stopping HDFS Services"

Task 2   Stopping Cloudera Manager Server

Follow this procedure to stop Cloudera Manager Server.

1. Log in as root to the node where Cloudera Manager runs (initially node03).

   Note:

   The remaining tasks presume that you are logged in to a server as root. You can enter the commands from any server by using the dcli command. This example runs the pwd command on node03 from any node in the cluster:

   # dcli -c node03 pwd
   

2. Stop the Cloudera Manager server:

   # service cloudera-scm-server stop
   Stopping cloudera-scm-server:                              [  OK  ]
   

3. Verify that the server is stopped:

   # service cloudera-scm-server status
   cloudera-scm-server is stopped
   

After stopping Cloudera Manager, you cannot access it using the web console.

Task 3   Stopping Oracle Data Integrator Agent

If Oracle Data Integrator Application Adapter for Hadoop is installed on the cluster, then stop the agent.

1. Check the status of the Oracle Data Integrator service:

   # dcli -C service odi-agent status
   

2. Stop the Oracle Data Integrator agent, if it is running:

   # dcli -C service odi-agent stop
   

3. Ensure that the Oracle Data Integrator service stopped running:

   # dcli -C service odi-agent status
   

Task 4   Dismounting NFS Directories

All nodes share an NFS directory on node03, and additional directories may also exist. If a server with the NFS directory (/opt/exportdir) is unavailable, then the other servers hang when attempting to shut down. Thus, you must dismount the NFS directories first.

1. Locate any mounted NFS directories:

   # dcli -C mount | grep shareddir
   192.0.2.1: bda1node03.example.com:/opt/exportdir on /opt/shareddir type nfs (rw,tcp,soft,intr,timeo=10,retrans=10,addr=192.0.2.3)
   192.0.2.2: bda1node03.example.com:/opt/exportdir on /opt/shareddir type nfs (rw,tcp,soft,intr,timeo=10,retrans=10,addr=192.0.2.3)
   192.0.2.3: /opt/exportdir on /opt/shareddir type none (rw,bind)
        .
        .
        .
   

   The sample output shows a shared directory on node03 (192.0.2.3).

2. Dismount the shared directory:

   # dcli -C umount /opt/shareddir
   

3. Dismount any custom NFS directories.

Task 5   Stopping the Servers

The Linux shutdown -h command powers down individual servers. You can use the dcli -g command to stop multiple servers.

1. Create a file that lists the names or IP addresses of the other servers in the cluster, that is, not including the one you are logged in to.

2. Stop the other servers:

   # dcli -g filename shutdown -h now
   

   For filename, enter the name of the file that you created in step 1.

3. Stop the server you are logged in to:

   # shutdown -h now
   

Task 6   Stopping the InfiniBand and Cisco Switches

To stop the network switches, turn off a PDU or a breaker in the data center. The switches only turn off when power is removed.

The network switches do not have power buttons. They shut down only when power is removed

To stop the switches, turn off all breakers in the two PDUs.

2.8.3 Starting Oracle Big Data Appliance

Follow these procedures to power up the hardware and start all services on Oracle Big Data Appliance.

Task 1   Powering Up Oracle Big Data Appliance

1. Switch on all 12 breakers on both PDUs.

2. Allow 4 to 5 minutes for Oracle ILOM and the Linux operating system to start on the servers.

3. If password-based, on-disk encryption is enabled, then log in and mount the Hadoop directories on those servers:

   $ mount-hadoop-dirs
   Enter password to mount Hadoop directories: password
   

If the servers do not start automatically, then you can start them locally by pressing the power button on the front of the servers, or remotely by using Oracle ILOM. Oracle ILOM has several interfaces, including a command-line interface (CLI) and a web console. Use whichever interface you prefer.

For example, you can log in to the web interface as root and start the server from the Remote Power Control page. The URL for Oracle ILOM is the same as for the host, except that it typically has a -c or -ilom extension. This URL connects to Oracle ILOM for bda1node4:

http://bda1node04-ilom.example.com

Task 2   Starting the HDFS Software Services

Use Cloudera Manager to start all the HDFS services that it controls.

1. Log in as root to the node where Cloudera Manager runs (initially node03).

   Note:

   The remaining tasks presume that you are logged in to a server as root. You can enter the commands from any server by using the dcli command. This example runs the pwd command on node03 from any node in the cluster:

   # dcli -c node03 pwd
   

2. Verify that the Cloudera Manager started automatically on node03:

   # service cloudera-scm-server status 
   cloudera-scm-server (pid  11399) is running...
   

3. If it is not running, then start it:

   # service cloudera-scm-server start
   

4. Log in to Cloudera Manager as the admin user.

   See "Managing CDH Operations Using Cloudera Manager".

5. In the Status pane of the opening page, expand the menu for the cluster and click Start, and then click Start again when prompted to confirm. See Figure 2-9.

   To navigate to this page, click the Home tab, and then the Status subtab.

6. On the Command Details page, click Close when all processes are started.

7. In the same pane under Cloudera Management Services, expand the menu for the mgmt service and click Start.

8. Log out of Cloudera Manager (optional).

Task 3   Starting Oracle Data Integrator Agent

If Oracle Data Integrator Application Adapter for Hadoop is used on this cluster, then start the agent.

1. Check the status of the agent:

   # /opt/oracle/odiagent/agent_standalone/oracledi/agent/bin/startcmd.sh OdiPingAgent [-AGENT_NAME=agent_name]
   

2. Start the agent:

   # /opt/oracle/odiagent/agent_standalone/oracledi/agent/bin/agent.sh [-NAME=agent_name] [-PORT=port_number]
   

2.10.1 About Predefined Users and Groups

Every open-source package installed on Oracle Big Data Appliance creates one or more users and groups. Most of these users do not have login privileges, shells, or home directories. They are used by daemons and are not intended as an interface for individual users. For example, Hadoop operates as the hdfs user, MapReduce operates as mapred, and Hive operates as hive.

You can use the oracle identity to run Hadoop and Hive jobs immediately after the Oracle Big Data Appliance software is installed. This user account has login privileges, a shell, and a home directory.

Oracle NoSQL Database and Oracle Data Integrator run as the oracle user. Its primary group is oinstall.

Table 2-9 identifies the operating system users and groups that are created automatically during installation of Oracle Big Data Appliance software for use by CDH components and other software packages.

2.10.2 About User Authentication

Oracle Big Data Appliance supports Kerberos security as a software installation option. See Chapter 3 for details about setting up clients and users to access a Kerberos-protected cluster.

2.10.3 About Fine-Grained Authorization

The typical authorization model on Hadoop is at the HDFS file level, such that users either have access to all of the data in the file or none. In contrast, Apache Sentry integrates with the Hive and Impala SQL-query engines to provide fine-grained authorization to data and metadata stored in Hadoop.

Oracle Big Data Appliance automatically configures Sentry during software installation, beginning with Mammoth utility version 2.5.

2.10.4 About On-Disk Encryption

On-disk encryption protects data that is at rest on disk. When on-disk encryption is enabled, Oracle Big Data Appliance automatically encrypts and decrypts data stored on disk. On-disk encryption does not affect user access to Hadoop data, although it can have a minor impact on performance.

Password-based encryption encodes Hadoop data based on a password, which is the same for all servers in a cluster. You can change the password at any time by using the mammoth-reconfig update command. See Oracle Big Data Appliance Owner's Guide.

If a disk is removed from a server, then the encrypted data remains protected until you install the disk in a server (the same server or a different one), startup the server, and provide the password. If a server is powered off and removed from an Oracle Big Data Appliance rack, then the encrypted data remains protected until you restart server and provide the password. You must enter the password after every startup of every server to enable access to the data. See "Starting Oracle Big Data Appliance".

On-disk encryption is an option that you can select during the initial installation of the software by the Mammoth utility. You can also enable or disable on-disk encryption at any time by using either the mammoth-reconfig or bdacli utilities.

2.10.5 Port Numbers Used on Oracle Big Data Appliance

Table 2-10 identifies the port numbers that might be used in addition to those used by CDH.

To view the ports used on a particular server: 

1. In Cloudera Manager, click the Hosts tab at the top of the page to display the Hosts page.

2. In the Name column, click a server link to see its detail page.

3. Scroll down to the Ports section.

2.10.6 About Puppet Security

The puppet node service (puppetd) runs continuously as root on all servers. It listens on port 8139 for "kick" requests, which trigger it to request updates from the puppet master. It does not receive updates on this port.

The puppet master service (puppetmasterd) runs continuously as the puppet user on the first server of the primary Oracle Big Data Appliance rack. It listens on port 8140 for requests to push updates to puppet nodes.

The puppet nodes generate and send certificates to the puppet master to register initially during installation of the software. For updates to the software, the puppet master signals ("kicks") the puppet nodes, which then request all configuration changes from the puppet master node that they are registered with.

The puppet master sends updates only to puppet nodes that have known, valid certificates. Puppet nodes only accept updates from the puppet master host name they initially registered with. Because Oracle Big Data Appliance uses an internal network for communication within the rack, the puppet master host name resolves using /etc/hosts to an internal, private IP address.

2.11.1 About Oracle Audit Vault and Database Firewall

Oracle Audit Vault and Database Firewall secures databases and other critical components of IT infrastructure in these key ways:

• Provides an integrated auditing platform for your enterprise.

• Captures activity on Oracle Database, Oracle Big Data Appliance, operating systems, directories, file systems, and so forth.

• Makes the auditing information available in a single reporting framework so that you can understand the activities across the enterprise. You do not need to monitor each system individually; you can view your computer infrastructure as a whole.

Audit Vault Server provides a web-based, graphic user interface for both administrators and auditors.

You can configure CDH/Hadoop clusters on Oracle Big Data Appliance as secured targets. The Audit Vault plug-in on Oracle Big Data Appliance collects audit and logging data from these services:

• HDFS: Who makes changes to the file system.

• Hive DDL: Who makes Hive database changes.

• MapReduce: Who runs MapReduce jobs that correspond to file access.

• Oozie workflows: Who runs workflow activities.

The Audit Vault plug-in is an installation option. The Mammoth utility automatically configures monitoring on Oracle Big Data Appliance as part of the software installation process.

2.11.2 Setting Up the Oracle Big Data Appliance Plug-in

The Mammoth utility on Oracle Big Data Appliance performs all the steps needed to setup the plug-in, using information that you provide.

To set up the Audit Vault plug-in for Oracle Big Data Appliance: 

1. Ensure that Oracle Audit Vault and Database Firewall Server Release 12.1.1 is up and running on the same network as Oracle Big Data Appliance.

   See Also:

   Oracle Audit Vault and Database Firewall Installation Guide

2. Complete the Audit Vault Plug-in section of Oracle Big Data Appliance Configuration Generation Utility.

3. Install the Oracle Big Data Appliance software using the Mammoth utility. An Oracle representative typically performs this step.

   You can also add the plug-in at a later time using either bdacli or mammoth-reconfig. See Oracle Big Data Appliance Owner's Guide.

When the software installation is complete, the Audit Vault plug-in is installed on Oracle Big Data Appliance, and Oracle Audit Vault and Database Firewall is collecting its audit information. You do not need to perform any other installation steps.

2.11.3 Monitoring Oracle Big Data Appliance

After installing the plug-in, you can monitor Oracle Big Data Appliance the same as any other secured target. Audit Vault Server collects activity reports automatically.

The following procedure describes one type of monitoring activity.

To view an Oracle Big Data Appliance activity report: 

1. Log in to Audit Vault Server as an auditor.

2. Click the Reports tab.

3. Under Built-in Reports, click Audit Reports.

4. To browse all activities, in the Activity Reports list, click the Browse report data icon for All Activity.

5. Add or remove the filters to list the events.

   Event names include ACCESS, CREATE, DELETE, and OPEN.

6. Click the Single row view icon in the first column to see a detailed report.

Figure 2-10 shows the beginning of an activity report, which records access to a Hadoop sequence file.

Figure 2-10 Activity Report in Audit Vault Server

Description of "Figure 2-10 Activity Report in Audit Vault Server"